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(S) Methods and apparatus for making and using distributed applications. 

@ A dient-server system for which applications 

programmers may easily write services and in FIG. 1 

which a relationship between a server and a 
sen/ice may be changed without halting the 
server. Both dient and server have access to 
copies of code for the sen/ice. The code has two 
parts : a caller portion which requests a sen/ice 
and a callee portion which executes the service. 
State variables in the dient process and the 
server process detennine which portion of the 
code is executed. This mechanism permits a 
sen/er to forward execution of the service to 
another server. The code for the sendee is 
written using a template which relieves the 
applications programmer of the need to write 
spedalized code. The server provides the dient 
with a server namespace which is distinct from 
the server's system namespace. The dient can 
locate a service by means of a service pathname 
in the system namespace. The server further 
provides the dient with namespace manipu* 
lation services which permit the dient to add 
services to and remove services from a server 
and otherwise to manipulate the server names- 
pace without halting the server. 
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1 Background of the Invention 
1.1 Field of the Invention 



The invention concerns data processing systems generally and more particularly concerns techniques for mak- 
ing applications programs to be executed in distributed computing systems and executing such programs. 

1.2 Description of the Prior Art 

Modern computer systems are often distributed, that is. the system is made up of a number of computers which 
are connected by a network. Each computer is capable of operating independently, but many tasks require 
the cooperation of several computers. For example, in many such systems, a first process running on one of 
the computers executes a program which controls a large disk drive: when a second process on another of 
the computers executes a program which needs a copy of a file on the disk drive, the second process requests 
that the first process send it the copy via the network. 

One way in which processes running in a distributed system may cooperate is as clients and servers. Ser- 
ver processes perform services for dient processes. To have a service performed, the client process sends 
a message requesting the service to the server process: the server process then performs the service and 
returns a message with the result to the client process. Thus, in the above example, the second process is a 
file server process, the first process is a client of the file server process, and the service is providing a coov 
of a file. 

A simple model for communications between dients and servers is the remote procedure call. In this mod- 
el, the dient calls a service in the server in the same way in which it would call a procedure which it executes 
itself, that is, the dient ceases executing the procedure from which it made the call, executes the called pro- 
cedure. and continues execution of the calling procedure on return from the call. In the same way, the process 
which makes the remote procedure call ceases executing the procedure which made the call. However, since 
the service is remote, the call turns into a message to the server. The server then executes the service spe- 
cified in the message and returns a message with the result to the dient When the dient receives the message 
with the result it resunws execution of the procedure which made the call. 

While the dient-server model for cooperating processes and the remote procedure call are both widely 
accepted in the computer arts, it remains difficult for the ordinary applications programmer to write programs 
using the model. There are several sources for the difficulty. First there are the complexities of communication. 
For example, the messages sent through the communications system often have representations of the data 
which are different from those used in the dient and the server, consequently, data must be encoded and de- 
coded each time a message is sent 

Second, there are the complexities of identifying the service remotely. The dient and the server are dif- 
ferent processes, and they may be running on different computer systems. When this is the case, each process 
may have a different environment for names representing entities such as files. Such environments are termed 
herein namespaces. When the dient and the server have different namespaces, the dient must know the name 
for the service in the server's namespace. 

Third, there are the complexities of binding, that is. relating the name of a service to the code for the ser- 
vice. The binding may be done statically, that is. it cannot be changed once the object code for the server has 
been produced, or it may be done dynamically. In the second case, the binding may be done when the server 
begins execution or during execution. Binding during execution is the most complex kind of binding, but also 
the most useful, since it permits addition of services to and removal of services from a running server. It thereby 
becomes possible to maintain services in a server without shutting the server down. 

Fourth, there are the complexities of service location. In order to achieve a higher degree of fault tolerance 
or to balance system load, it is often useful to be able to have different servers execute the service for the 
dient at different times. However, this needs to be a done in a way which requires no change in the way the 
client calls the service. 

These complexities of communications, naming, binding, and location have at least two unfortunate con- 
sequences: first they leave the applications programmer with the the choices of making do with a set of pre- 
defined services which are provided by the operating system or undertaking the enormous effort required to 
write a dient-server system from scratch. Second, even if the applications programmer does undertake the 
required effort, the result is usable only on the systems for which it is written. What is required, and what the 
techniques disclosed herein provide, is a way of implementing services which is no more difficult than imple- 
menting an ordinary function and which produces reusable implementations. 
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2 Summary of the Invention 

The invention provides users of computer systems with techniques for implementing client-server systems 
which offer the following advantages: 
5 • it is as easy to implement a service as it is to impiemcnt a function in a standard programming language; 

• it is as easy for a client to specify a service in a server as it is for the client to specify a program in the 
client's own namespace: 

• the dient may add services to and delete them from the server and otherwise manipulate the server's 
namespace; 

10 • services may be added to or removed from a server without interrupting operation of the server and 

• a server may either provide a service itself or forward the service to another server. 

Other objects and advantages of the apparatus and methods disclosed herein will be apparent to those 
of ordinary skill in the art upon perusal of the following Drawing and Detailed Description, wherein: 

75 3 Brief Description of the Drawing 

FIG. 1 is a diagram of a system of clients and servers which employs the techniques disclosed herein; 
FIG. 2 shows code for a service; 
FIG. 3 shows how code for a service is produced: 
20 FIG. 4 shows how a service is called: 

FIG. 5 shows how a service is forwarded; 

FIG. 6 shows predefined services provided by a server 

FIG. 7 shows a super server; 

FIG. 8 shows the data structure used to define a server's namespace; 
25 FIG. 9 shows how a server can be used to achieve fault tolerance; 
FIG. 10 shows details of a dient; 
FIG. 11 is a first detailed diagram of a server and 
FIG. 12 is a second detailed diagram of a server. 

Reference numbers in the Drawing have two parts: the two least-significant digits are the number of an 
30 item in a figure; the remaining digits are the number of the figure in which the item first appears. Thus, an item 
with the reference number 201 first appears in FIG. 2. 

4 Detailed Description of a Preferred Embodiment 

55 The following DetaHed Description begins with an overview of a system in which a presently- preferred embodi- 
ment of the invention is employed, continues with an overview of the dient and server, then provides details 
of how services are implemented, of how a server namespace is implemented, and of how server namespace 
operations are implemented, and finally describes a number of applications of the preferred embodiment 

I 

^ 4.1 System Overview: FIG. 1 ' 

FIG. 1 provides an overview of a preferred embodiment Distributed system 102 is made up of a number of 
computer systems 101 (a.b.c.and d) are connected by communications system 105. Each computer system 101 
has a processor upon which one or more processes may be executed and a file system 103 in which files con- 
taining programs and data may be stored for use by the processes. When a process running on a system 101 
wishes to access a file in file system 103, it uses a name for the file in 101's system nanne space. There are 
four processes shown in FIG. 10. a dient process 107 and server processes 131 (a.b, and c) (henceforth, simply 
dient 1 07 and server 1 31 ). In FIG. 1 0. the dients and servers are on separate systems 101 . but a given system 
101 may indude both clients 107 and servers 131. a server may have many dients. and the server and its 

50 clients may be on the same or different systems. Further, one server process 131 may be a client process 
107 for another server process 131. 

Server 131(a) provides three services 116 named A, B. and C to dient 107; the services appear in FIG. 1 
as service A 116(1), service B 116(2). and service C 116(3). There is service code 117 corresponding to each 
service 116. Code 117(1) for service A 116(1) is located on file system 103(c); code 117(2) for service B 116(2) 

55 is located on file system 103(b); code 11 7(3) for service 116(3) is located on file system 103(d). Furthermore, 
file system 103(a) has a copy of the code for each of services 117(1..3). As will be explained in more detail 
later, service code 117 indudes code to be executed by a caller (client 107 or a server 131 operating as client 
107's agent) and a callee (the server 131 which actually performs the service). Client 107 is able to locate 

3 
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37!hLh L'l? """k? °'«rverlist 118(a) in file system 103(a). Server list 118(a) has an entry for each server 
wh e M "tst^^^^^^ ''^ ""^^ " -™ °' a system 

wnicn Client 107 ,s running. Since the system name is part of the system name space of computer svstem 101 

vel ?31 anr.h! f^i detail below, servers 131 are also accessible from other ser- 

c::.;fe;r:;^haV;i%rr3'^^ 

Se v?r 1 3l7a) ^^^^^^^^^^ ' ""^^ ''^ -o*"- server' 3^ 

Serve 131(a) can e ther provide the service 116 Itself or tofKrartf the dient lOrs request for the service 116 

amespari73%^^^^^^^^^^^^ 

rhr«r„t» 2^ . correspondence between the name of the service and a copy of code 117 for 

the service which s ,n system 101(c). If another server, say 131(b). provides the service 116 nJLl 
space 133 for server 131(a, addittonally establishes a corresponded,^ bJ?::en:he ame c t^^ 

and,tsname.nthenamespace133forserver131(b).lnFIG.1.service116(1)hasthena4rserv^^^^^^ 
has the name 8. and service 116(3) has the name C. In a preferred embodiment. serverT3i(a) r^^^^ 

aT. ^ JT' "'T"" ' ^presenting the root o/hSriJi 

**=°'"P"ter_system_name>/<server_name>/<service name> 

TJ^-,T. ' ^ ^ service Aby means of the service pathname /condor/dcs/A. Similar^, the other server, 
vicls avan",^^ namespaces 1 33 (not shown in FIG. 10) to the clients and servers to which they make s.^ 

. J"ii7f embodiment, dient 107 exectites a service 116 by executing the callee portion of service 

Z^^:T Z '"f '""J?'? "6 in server 131(a). The arguments for the funrtionfm a 

IXTJl^^ n "^"^ " ••^"^ « P«'"'" to « list of arguments. anJ 

abuffer for the result of the service call. IfdientlOT uses ServiceCall to indi^^^^ 
to be provided wrth service 116 B. then the pathname pointed to by the first variable is /cond JdcSs 
..r.lr'!^T^ ServiceCall by dient 107 results in a message being sent to system 101(c) which specifies 

31 a rlJivL'^K °' « ^"'^ ^rS""^"' When server 

131(a) receives the message, it uses the service pathname to look up the service in server namespace 133 

TnToJZTtT- T"-" " '° ^^^^^ "^•""P-" 133 contains a 

otherservtMSI '"^ ''^ "^^^P^^** ^^3 belonging to the 

is ioZZ^ZlTll^ simply executes the portion of code 117 which 

exltU Thf 'S T w ' ""'""S* '""^ 10^- ^^^'^^ "'"P'etes 

executKDn of the part of code 11 7 which is to be executed in dient 107. If another server 131 is to provide the 

oZLTZ-JTJ ""^'^ « '° "^'^^'^l ""'"'es the link 

pathname provKled by server namespace 133 in ServiceCall along with the arguments whid, server 131(a) 
r^e^ed from dient 07. In the case of service B. server namespace 133 indicates that the service 116 is pS 
ni/M \ r k" " "^^'^S* P™''""'^ "y ServiceCall is forwarded to server 

to server 131(a). which ,n turn returns it to dient 107. As is apparent from the foregoing, the message produced 
by the invocation of ServiceCall in dient 107 may be forwarded any number of times. 

4.2 Service Code 117: FIG. 2 

Each service 116 is defined by service code 11 7; as indicated above, each dient 107 which can request a ser- 
vice 116 has access to a copy of service code 117 for the service 116: similarly, each server 131 which can 
PfOv.de the service 116 has access to a copy. If dient 107 and server 131 are in the same system 101. they 
both may use the same copy of service code 117. In a preferred embodiment, service code 117 used by dient 
107 o stetically linked to the code executed by dient 107 which calls the service 116 implemented by service 

! I I . ' r ^'^^ '"'y P' 't "^ay «'so be dynamically linked, 

as will be described in more detail below. Dynamic linking of service code 117 permits services 116 to be added 
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to or removed from server 131 without interruption of operation of server 131.ln other embodiments, service 
code 117 used by client 107 may also be dynamically linked. 

FIG. 2 shows details of service code 117; in a preferred embodiment, it has 6 parts: 

• Argument converter 201 is code for encoding the arguments for the service 116 into a form suitable for 
5 use by communications system 1 05 and decoding the arguments as received from communications sys- 
tem 105 into a form suitable for use by dient 107 or service 131; 

• Result converter 203 is code which does the same for the results of the execution of service 116. 

• Init 205 is code which is executed when service code 117 is installed in a server 131; 

• Function 207 is code for the function performed by the service 116 implemented in service code 117; 
w it includes two subparts: 

- Caller code 210. which is executed by a client 107 or a server 131 which is calling a server 131; and 

- Callee code 213. which is executed by server 131 which actually provides the service. 
Converters 201 and 203 are necessary because the networks used in modern distributed systems typically 
represent data as a stream of bytes. Consequently, the arguments for the service 116 and the results from 

IS the service 116 must be translated between the forms which are required in the systems in which service 116 
executes and the fonms required by the network. In the preferred embodiment the code for argument converter 
201. result converter 203. and init 205 is provided automatically when service code 117 is generated; the person 
writing service code 117 need only supply the code in function 207. 

Caller and callee code are written in a preferred embodiment as branches of an if statement The function 

20 used in the if statement lsCaller() 209, is a special function used to implement system 1 02. It returns one value 
if service code 117 is being executed by a caller and another if service code 117 is being executed by a callee. 
In the case of execution by a client 107. the function always returns the caller value: in the case of a server 
131. the function returns the callee value unless the server is forwarding the service request in that case, it 
returns the caller value. 

25 

4.3 Writing Service Code 117: FIG. 3 

FIG. 3 shows how Service Code 117 is written in a preferred embodiment There are four steps: 
30 • Specifying Service Interface 

The preferred embodiment employs an extension of a remote procedure language used in Sun computer 
systems to specify the service interface. The specification defines data structures of input arguments and 
output results in a C-like syntax. It also declares a service function with the syntax: 
35 -service" type-specifier identifier "(• type-specifiers •)";' 

where "service" is a keyword, the first "type-specifier' specifies the type of the value returned by the service 
function, the "identifier' specifies the name of the function and the "type-specifiers* in parentheses specify 
tne types of the function's arguments. 

-to « Generating RPC Stub and a Service Template 

Service code 117 is written using a service compiler, called ServiceGen 303, which takes a service inter- 
face as input (i.e.. spec.x 305. and generates three outputs: XDR routines (i.e.. the file xdr.c 307), which are 
argument converter 201 and result converter 203 In a preferred embodiment a template (i.e.. the file template.c 
•*5 309) which a programmer can edit to implement the service function; and an initial function (i.e., the file m/tc 
311) which contains code used when the service code 117 is installed in a server 131. 

• Implementing the Service Function 

50 application programmers implement the service function by editing (313) template 309 to produce the code 

for the service function servicex 315. 

• Creating a Service Library 

53 Service code 117 is generated by compiling and linking service function 315. which becomes function 207. 

xdr routines 307, which become argument converter 201 and result converter 203, and initial function 311. 
which becomes init code 205. Switches on the compiler permit compilation of service code 117 in forms which 
permit its incorporation into the server or client program at link time, at system initializatkjn time, and at run 

5 
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time. 



4.4 Exampio of lmpl«m«nting S«rvie« Cod« 117 

The following example will show how service implementation system 301 may be used to implement a service 

?or me 2 r„,;'T t! '"^ ' P^"'"""* '"^ f »e and anotheJpat ame 

for the dest.nat.on f lie. The result returned by the cp() function is an integer, with the value 0 if the exe<S^n 
successes otherwise a system error number (e.g..errno in UNIX). n me value o if the execution 

The service interface for cpQ is the following. 



struct IiLArguaent 
{ 

char froaCl28]; 
char to [128]; 

>: 

(a). service iat cpClnArgnaent) ; 



6 
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10 



15 



20 



bool.t XdrlnArguaentO; /* XDR Function for input: InArguaent •/ 
bool.t XdrlntO; /. XDR Function for output: int •/ 

int init (pathname) 
char 'pathnaae 
caddr.t cp() ; 



/• service function «/ 



int init (pathname) 



{ 



if (creatnode (pathname. SERVICE, cp. XdrlnArguaent . Xdrint)) 
return (0) ; 

else 

return (-1); 



25 



30 



This function is invoked by server 131 when the service 116is linked by server 131. It calls a function, crea- 
tnode(). which puts the name cp into server namespace 133. As will be explained in more detail later, each 
name in namespace 133 is represented by a node, and creatnodeQ creates such nodes. The function takes 
five arguments: the part of the service pathname following the server name, the type (SERVICE, LINK, or DI- 
RECTORY) of the node, a function pointer to function code 207, a function pointer to argument converter 201 . 
and a function pointer to result converter 203. 
Template 309 for cp() looks like this: 



35 



40 



45 



50 



55 



7 
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int* cpCargp, objp) 

struct laArgunent 'argp; 
char 'Objp; 



10 



static int res; 



15 



20 



25 



if (IsCallerO) 
{ 

if (ServiceCalKobjp, argp, ftres) !»SUCCESS) 
return (NULL); 

else 

return (&rea) ; 

} 

/* implement the service here*/ 
ret urn (Ires); 



30 



35 



40 



As previously indicated, service function 207 indudes two parts: caller code 210 and callee code 213 Each 
t ^ ' H °' T- '''' ^"^^^^^ the f unction IsCal- 

207 ,s be,ng invoked by a caMeror callee and also determines whether caller code 210 or callee code 213 is 
executed In the template, the code marked by • in the left margin is caller code 210: callee code 213 remair^s 
0 be .mf^emented at the point indicated by the comment, r implement the service hereV. Caller code 2lS 
contains Serv.ceCall function 210. which actually invokes the service 207 on a server 131 

As can be seen from the above, the template makes implementing a service function 207 as easy as im- 
p^emen ,ng ariy other function. The f ully-implemented service function 207 looks like this (lines beginnL with 
* are the callee code 21 3 which has been added to the template)- 



50 



55 



8 
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int« cpCargp. objp) 

struct InArguaent 'argp; 
c^ar •objp; 

{ 

static int res; 
iat rfd; 
int wfd; 
int length; 
char bufCl024]; 



20 it (IsCallerO) 

{ 

if (ServiceCalKobjp, argp, *res) !«SUCCESS) 
return (NULL); 

else 

return (Ares); 

30 



35 



40 



45 



50 



55 



9 
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if (Crfd = open(argp->from. O.ROONLy)) < 0 || 

(vfd = op«(argp->to, O.CREAT | O.WRONLY. 0600)) < 0) 



10 



30 



35 



40 



SO 



res 3 ermo; 
return (Ares); 



15 



while (ClengtH « read(rfd, buf , 1024)) > 0)) 
20 vriteCvfd, buf, length); 



55 



4 



close(rfd); 
25 ^ close(irfd); 



res = 0; 
retumCires); 



tions'rn^thr;^.^J,\' ' ^'^^ ^"^^'^^^ ^^^vlce function, XDR f unc- 

tions. and tne initial function as previously described. 

4.5 Implementation of Client 107: FIG. 10 

As previously explained, both client 107 and server 131 have a codv of sArvirA • * . 

ponion 207 Of service codeiu^as caller code wt,ich is ^ 

bodmjent executes object code for an applications program which uses the service imfJementedT^^^^^ 

lerO function 209, object code for the ServiceCall function, and object code for a sys tern-DrovS^ rl^^l b1 
cedure call. FIG. 10 shows the relationships between the parts of ervS o^i 117 and ,heT^^^^ 
Of Client 107. IsCaller 1005 is the object code for lsCaller() function 2S-^e value rt rl^^^^^^^^ 
vanable ME 1007. In client 107, ME 1007 always indicates a callee Serv ceCa7lO 6^^^^^^^^^ 
ServiceCal. function 211: data used by Serviceil, 1016 includt^^gsToTa'?,!; '/a^^^^^^^^ 

e ifv^lisMlV":- h'"? ' '^'^ ^ e^-tw'h1 

service nb. and server list 118. which contains server name 1013 for each server acce»ihi« Hionf iat 

and server location 1015 for the server. Server location 1015 is a network add^ s forThr^^^^^^^^ 

n.cafons system 105. Remote procedure call 1017. finally, is the remote procedure c^ lOuXiTeS^Th^ 

funct-on 1016. and remote pmcedure call 1017togethermake up caller stub 1023ofdientl07 As will be see^^ 
in more detail below, servers 131 also have caller stubs 1023 'u' mswi.i oeseen 

the fu„Tn^°„?' " <="«"^ 1 07 invokes the service using 

s ^he exltirjr T'"*"'"^'"'' '''''' specif icatton 303: the result of the invoca ion 
r^lfesC 1 .? " 207. Since client 107 is executing service function 207. IsCaller 

.ndeates that the caller is executing the service function, and the code in caller portion 210 is executed. That 

10 
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code contains an invocation 211 of ServiceCall 1016. The invocation specifies service pathname 116. which, 
as previously indicated, includes the system 101 upon which server 131 executes, the name ofserver131 and 
the pathname of service 116 in server 131*s namespace, arguments 1003 for the service 116. and buffer 1004 
for the results of the execution of the service by the server. For example, if the system name is condor and 

5 the server name is dscs and the pathname in dscs's server nanrespace 133 /bin/cp. the service pathname 
will be /condor/dscs/bin/cp. 

Service call 1016 places the arguments for service 116 in Args 1003, uses the server name in the path- 
name to find the network address for the server in server list 1 1 8. and then invokes remote procedure call 1017. 
The invocation specifies as arguments the network address, the pathname for the service in the server name- 

10 space 133, arguments 1003.buffer 1004 for the result, argument converter 201, and result converter 203. Re- 
mote procedure call 1017 uses argument converter 201 to put the arguments for service 116 into the proper 
form for the communications system, then makes a call message addressed to server 131 which contains the 
service path name and the encoded arguments, and finally sends the call message 1019 to server 131. 
When server 131 is done executing callee code 213. it sends a return message 1021 to RPC 1017 which 

IS contains the results. RPC 1017 uses result converter 203 to decode the results, places the decoded results 
in results buffer 1004, and returns to ServiceCall. which in turn returns to caller portion 210 of service function 
207. Execution of caller portion 210 is then completed. Typically, completion of execution of caller portion 210 
involves returning the contents of results buffer 1004 to the execution which invoked service function 207. 

20 4.6 Implementation of Server 131 : FIG. 11 

Fig. 11 shows details of those partsof the implementation of server 131 which are directly involved in the exe- 
cution of a service 116. Server 131 has its own copy of service code 117 for the service 116 and additionally 
includes a caller stub 1023 with the same components as caller stub 1023 in client 107. Caller stub 1023 and 

25 the other components of server 131 which are directly involved in the execution of service 116 together make 
upcalleestub 1103. The nwst important additional component of callee stub 1103 is dispatch 11 05, object code 
for a function which responds to a call message 1019 from a client 107 by executing service function 207 for 
the service 116 specified in the call message. 

Continuing with a more detailed description of the implementatton of server 131. the call message 1019 

30 received in dispatch 1105 specifies the service pathname in server namespace 133 of service code 117. dis- 
patch 1105 invokes a function. FindService. in server namespace 133. which takes the service pathname 1117 
for the service and returns information 1119 specifying where the service is to be provided. Next, dispatch 
1105 invokes decode function 1109. which takes the message and a pointer to argument converter 203 and 
decodes the arguments in the message and places them in Args 1115. Thereupon. dispatch1105 determines 

35 whether information 119 specifies that the service is to be provided by server 131 or another server 131*. In 
the latter case, the information includes a link pathname for the service which specifies the service pathname 
for the service 1 1 6 in server namespace 1 33* belonging to server 131 \ When the service 11 6 is to be provided 
by server 131. dispatch 1105 sets state variable ME 1111 to specify a callee: otherwise dispatch 1105 sets 
state variable ME 111 to specify a caller. Thereupon, dispatch 1105 invokes service functwn 207 with the ar- 

-to guments in Args 1115 and a service pathname, as shown by arrow 1121. If the service 116 is to be provided 
in server 131, the service pathname is the pathname for the service 116 in server 131's server namespace 
1 33; if it is to be provided in server 1 3 V. it is the pathname for the service 1 1 6 in server 1 31 "s server namespace 
133\ 

As described in the discussion of client 107. whether caller portion 210 or callee portion 213 of service 
^5 function 207 is executed depends on the value returned by IsCaller 1005 of caller stub 1023. which in turn 
depends of the value of ME 1111. Thus, if service location 1119 specifies a location on another server 131\ 
dispatch 1105 has set ME 1111 to indicate a caller, and caller portion 210 is executed with service location 
1119 and arguments 1115 as arguments for ServiceCall 1016. The invocation of ServiceCall1016 proceeds 
exactly as described for client 107, except that the results which ServiceCall 1016 receives from RPC 1017 
so are placed in results buffer 1113 and that the function which resumes execution on return from caller portion 
210 is dispatch 1105. as shown by arrow 1123. 

If information 1119 specifies that server 131 is to provide the service 116. dispatch 1105 has set ME 1111 
to indicate a callee. IsCaller so indicates, and callee portion 213. which contains the code which performs the 
actual function for the service, is executed using the contents of Args 113. The result of the executton is placed 
55 in Results 1113, and the return is to dispatch 1105. Regardless of whether caller portion 210or callee portion 
213 was executed, dispatch 1105 uses results converter 205 to encode results 113 and sends the encoded 
results in a return message 1021 to the source of the remote procedure call, which may be a dient 107 or an- 
other server 131. 
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4.7 Calling and Forwarding a Service 116: FIGS. 4 and 5 

S sl« «i?v ''"V'°' '^-^ namespace 133 thaTrservS 

116. se ver 131(.) .s an agentfor server 131(j). that is. that service 116 is to t)e executed by ter^\?m^ 
sequen y. callee stub 1103 sets ME vaHabie 1111 ,0 indicate -caHer* and when calS^^ ub XxecJ^L se^ 
ver 131(0 s copy of service code 1 17. IsCaller indicates -caller- and it is caller portion 210 of ser^w .SS^ m 
which ,s executed. Callee stub 1103 indudes a caller stub 1023 and a service^sTJls and v^e^S^c^Ll 
211 IS executed m caller portion 210. the msult is a call message 1019 for service 116 to se^er 13^ i^^^ 
server I31(j)-s server namespace 133 indicates that server 131(i) is to exe^te s^ v^« 1^6 TJ^er 13?« W 

user program 401 wh,ch invoked service 116. Of course, server 1310) may also be an agent, in whS case 

the length of the chain of agents: however, in a preferred embodiment the call message from client 107 in! 
dudes a value which limits the number of times the execution can be forwarded, and^hus p^vent, aZ^^^ 
for a service 116 from being forwarded around a loop of servers?i31. -: ,1- 

4.8 Implementation of Server Namespace 133: FIG. 12 i' -V " 

«mhoHiZ??' °' implementation of server namespace 133 for a server 131 in a preferred 

embodiment Server namespace 133 maps service pathnames onto location information 1119 for wrvi^ 
thetr 7 °' " '^^-^ ""^e Is to be p^vided by the ie ver l1?* 

upon which se ver 1 31 is running or the system path name of a file containing service code 117 In fUe system 
103belongmgtosystem101.lftheservice116is to be provided by anotherserver131^thelocat^^^^ 
IS the service path name of service 116 in that server 13rs servernamespace 133^ Conseq^en^^^^^ 

sll'Z nam f iJir- V''- ''''T '''' '''' ' namespa'ce St^" Z 
service path names 1117 into service location information 1119 

hierJ^SvT„rjr'' t "3 include hierarchy representation 1212. which represents the 

?eo»22^^,r ^T^r UP servernamespace 133. namespace primitives 1203. whichoperate on hierarchy 

03 aTd t^n^i^l • ''T. T '"^l '"^""^ "<^« 1" "l^k drive 
103 and the names specified m h«rarchy representation 1212. and namespace services 1201 which is a set 

s^r^rTK"''*^"""^'^''™^^^^^ 

aS^w 1202 "^-^^^P^" P"'"«'^« 1203 'o P«rfom, the actual manipulations, as shown by 

4.8.1 Hierarchy Representation 1212: FIGS. 12 and 8 

Continuing in more detail with hierarchy representation 1212. representation 1212 is a tree with three kinds 
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of nodes 1206: directory nodes 1207, which represent directories in server namespace 133. service nodes 
1 209, which represent services 116 which are executed by this server 131 , and agent nodes 1 211 , which rep- 
resent services 116 which are executed by other servers 131 for this server 131. FIG. 8 shows how nodes 
1206 are implemented: each node has two parts: a node structure 801. which represents the node itself, and 
context structure 803. which represents either the directory or the service 116 represented by node 1206. FIG. 
8 also has a detail of node structure 801; it contains name 804. which is the name in server namespace 133 
represented by node 1206. attributes 833, which indicates attributes of the node, parent pointer 807, which 
points to the parent of node 1206 in hierarchy representation 1212. and cont_ptr 809. which points to context 
structure 803 for the node. 

The attributes indude the node 1206's type in field 805. and in the case of service nodes 1209, they also 
include mapped bit field 835. which indicates whether the service node is mapped to a file containing service 
code 117, and if the node is not mapped, file attributes for service code 117. The file attributes indude the 
last time service code 117 was accessed by server 131. in field 837, the time the file containing service code 
1 1 7 was created, in field 839. and an access control list in field 841. The access control list is a list of the users 
allowed to access the service. 

The contents of a context structure 803 for a service are shown at 811 . The first three fields are set when 
service code 117 is loaded into the memory of system 101 upon which server 131 is running. func_ptr 813 is 
a pointer to function code 207 for the service: in_dec_ptr 815 is a pointer to argument converter 201 and 
out_dec_ptr is a pointer to result converter 203. SL path 819 is the system path name of service code 117 In 
file system 103. The system path name is used by service linker 1205 to locale service code 117 in file system 
103. SL.handle 821 is the value returned by file system 103 when service linker 1205 opens service code 
117. Linkjnfo 823. finally is used when node 1206 represents an agent 1211. In that case. Linkjnfo 823 con- 
tains a service path name for service 116 in server 13V which is to provide the service 116. 

The contents of a context structure 803 for a directory are shown at 825. The first field, map^path 827. 
is used to map a directory name in server nanf>espace 133 to a directory name in file system 103. When this 
has been done, server 131 automatically adds services 116 with service code 117 in the mapped directory to 
its server namespace 133. The field contains the system path name in file system 103 of the directory which 
corresponds to the directory in server namespace 133 to which context structure 803 belongs. The remainder 
of the fields in context structure 827 are pointers to the nodes 1 206 which represent the services or directories 
in the directory represented by dir_struc 825. There is a node_ptr 831 for each service or directory, and the 
list of pointers makes up nodejjtrjist 829. 

4.8.2 Details of Namespace Primitives 1203 

The primitive operations in server namespace 133 fall into two dasses: locating a node 1206 when given the 
path name for the entity represented by the node and adding and removing nodes 1206 in hierarchy represen- 
tation 1212. 

The primitive for locating a node is namei. nameitakes a pathname in server namespace 133 as as an 
argument and returns a pointer to node 1206 which represents the pathname. The function begins at the root 
of hierarchy representation 1212 and works down the pathname and hierarchy representation 1212 a compo- 
nent at a time. If namei either runs out of names in the pathname or runs out of nodes in representation 1212, 
or finds a service node 1209 or an agent node 1211. it returns the pointer to the current node 1206 and any 
remainder of the pathname. 

The primitive which adds a node is called creatnode. its arguments indude the type of the node and in- 
formation required to fill in the relevant fields of node structure 801 and context structure 803. The primitive 
creates a node structure 801, fills in fields 604, 833. and 807. aeates the proper type of context structure 
803, fills in fields of the context structure as required by the node type, and fills in field 809 of node structure 
801. The primitive which removes a node simply unlinks node structure 801 from hierarchical representation 
1212. updating node ^trjist 829 in the parent of the removed node 1206 as required. 

4.6.3 Using the Primitives 

The primitives are used to execute a service 116 and to perform namespace services 1201. As prevksusly de- 
scribed, executing a service is carried out in server 131 by dispatch1105. a component of callee stub 1103. 
dispatch receives a message with a service's path in name space 133 and uses the function FindService to 
find the service. FindService uses namei to move down hierarchical representation 1212 until it reaches a leaf 
node. 

What happens then depends on the node type. If the leaf node is a service node 1206 or a an agent node 
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827 pr„Ht " '""P'":*'"'"^ ""d* »° "^isP^ch. If the node is a directory node and has a map path 
BJ^^rnH J ! * °' P''""*'"* '° ' «"«<1 DLService. which used map path 

bLiln 7?°'^ t °' '^^ P^"' '° '°<=3te service code 117 

T: ' """"^y representation 1212 for ser- 

v.ce 116. Thereupon. ,t returns the service node 1209 for the service 116 to dispatch. In all other cases. Find- 
Service returns a value which indicates failure. 

Lmt hf tr^r^^^ , "^'"^ ""''"'"^ '"-'^"-P'' in service, true 

811 for the node 1206 to locate argument converter 201 in service code 117 and decode the argumentf^ the 

2 n«Jl I,nfi . ' '""^f 3"'^ the handle for the service to be called from link info field 823 
f node 1206 is a service node 1209. dispatchllOS sets me 111 to indicate callee. In t«th cases, dispatch 1105 

re e^d bv?;.'!'' r !! *"°'' '"""^^'""'^ -"^PP^-^ "5 is set. the invocation is 

cTm^m » ^ °' "'"'^^ •f'" =heck is made on access 

control l.st 841 in node structure 801 for the service 116. On return from the invocation, dispatch llOS^sel 

L" ;frf '^=f 203 in service code 11 7 and encode the results of the execution of the 
service. The encoded result is then returned to the caller. 

The namespace primitives are also used by namespace services 1201. FIG. 6 is a list of those services 
lirXT!^ e'"bo1i'"enL With the exception of null 621. each of the services 602 either reads or alters a 
node 1206. In a preferred embodiment, directory nodes 1207 are added by mkdir service 571. mkdir 671 takes 
the pathname m name space 133 of the new directory as an argument After name! has located the already- 
existing part of the pathname, creatnode simply makes node structure 801 and dir struc 825 fbr the new di- 

^ " ''""^""^ " '''^'^"y- « PO"*"' to 'he new 

node 1206 to nodej)trJ«t829,n the node 1206 which is the parentofthedirectory being creat^^ 
the pathname of the node to be removed as an argument At the time of the removal, the directory node 1207 
may not have any children. The remove primitive simply unlinks the specified node 1207 and updates 
node_ptrJist 829 in the parent node of the deleted node accordingly. 

The mkservice service 605 adds a service node 1209 to hiererohy representation 1212. The arguments 
for mkserv.ce are the service pathname of the new service in server namespace 1 33 and the system pathname 
of the fHe containing service code 117 in file system 103. mkservice uses name! to locate the directory node 
which IS the parent of service node 1209 for the service 116 to be added. Once the proper directory node 
1 207 has been located, mkservice uses the diopen primitive provided by service linker 1205 to open service 
code 117 and uses'another primitive, disym. provided by service linker 1205. to execute init portion 205 of ser- 
vice code 11 7 with the arguments of mkservice. Init portion 205 in turn uses creatnode to create the new node 
and fill in fields 804-809 in node structure 801 and service.struc 811. A service node 1209 is removed by the 
rmservice 607 service, which employs the removal primitive substantially as set forth above for directories 

The symlmk service 609 changes a service node 1209 to an agent node 1211. The arguments are the ser- 
vice pathname of the service 116 in the present server 133-s server namespace 133 and the service pathname 
of the service 116 in the server namespace 133 of the new server 133". symlink uses the service pathname 
of the service in the present server with namei to obtain a pointer to the service node 1209 and then changes 
the type of the service node 1209 to LINK and places the service pathname of the service in the new server 
133- in link info field 823 of service.struc 811. The unsymlink service 611 changes an agent node 1211 back 
into a service node. The service takes as its argument the pathname for the agent node 121 1 in server 133. 
It uses namei to locate the node and then resets the node's type to SERVICE. 

The bind and map services 613 and 615 establish relationships between nodes 1206 and parts of the sys- 
tem name space of system 101 upon which server 131 runs, bind 613 takes as its argument a service pathname 
in server namespace 133. It 613 uses namei and the service pathname to locate the node 1209 for the service 
It then sets mapped bit 835 to indteate that the service 1 16 has been bound to the file containing service code 
117. Mapped bit 835 can be reset by setstat as described below. 

The map service 615 maps a directory node 1207 in nierarchy representation 1212 onto a directory in file 
system 103. The arguments for map are the service pathname for directory node 1207 and the system path- 
name for the directory in file system 103 which is to be bound to directory node 1207. The service uses namei 
to locate the directory node 1207 and then wntes the system pathname for the directory in file system 103 
into map_path field 827. 

The Stat and dir namespace services 602 return information about a service 116 or a d^ectory. stat 602 
returns the attributes of service 116. dir returns a list of the contents of a directory in server namespace 133. 
Both services take a service pathname as an argument. Both use namei and the service pathname to locate 
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the node 1206 specified by the service pathname, and the information returned by the services is then ob- 
tained from the specified node. In the case of stat the information is always the node's type 805 and the setting 
of its mapped bit field 835 from node structure 801. If mapped bit field 835 indicates no mapping, the infor- 
mation further includes the contents of access time field 837. create time field 839. and access control list 
field 841. If mapped bit field 835 indicates mapping, this information is obtained from the file which contains 
service code 117. setstat623 permits mapped bit field 835 and the contents of fields 837-841 to be set by a 
client 107. The arguments are the service path name for the node and the values to which the attributes are 
to be set The function uses namei to locate the node 1206 and then sets the relevant fields from the values. 

getservice 625 and putservice 627 permit service code 117 to be uploaded from a client 107 to a server 
131 and downloaded from a server 131 to a dient 107. Uploading is done by putservice 627. which takes as 
its arguments a service path name for the service 116 and a buffer with service code 117. Server 131 performs 
the function by creating a file in its file system for service code 117 and creating a service node 1206 for the 
service 116 which specifies the file containing the uploaded service code 117. Downloading is done by get- 
service 625, which takes two arguments: the service path name for the service 116 and a buffer for service 
code 117. In server 131. getservice uses namei to locate service node 1209 for the service 116, uses the in- 
formation in service node 1209 to locate service code 117, and sends service code 117 back to dient 107. In 
client 107, the service code is moved from the buffer to a location in the system name space of dient 107 and 
is then dynamically linked to the code being executed by dient 107. 

4.9 Applications of Servers 131: FIGS. 7 and 9 

Servers 1 31 may be used in a number of ways to solve problems of distributed systems. One such problem 
is fault tolerance. As shown in system 901 of FIG. 9. A server 131(a) can forward a request fbr a service to 
any of a number of servers 131 (b..n). Consequently, server 1 31 (a) can respond to a failure by one of the ser- 
vers 131(b..n) by using symlink 609 to change the agent node 1211 for the service so that the the request for 
the service is forwarded to a working server 131. 

Forwarding requests for services can also be used to balance loads among servers 131{b..n). Server 
1 31(a) need only keep track of how many requests have been forwarded to each of the servers 131(b..n) and 
change the agent node 1211 as required. Such load balandng may of course also be responsive to other in- 
formation such as the state of the systems in which servers 131 (b,.n) operate or the tinne of day. In some em- 
bodiments, link info 823 can be a pointer to a scheduling functton which determines which of the servers 
1 31 (b..n) the service request is to be forwarded to. 

The fact that a service 1 1 6 is accessed by means of server namespace 1 33 also makes dynamic updating 
of services easy. All that need be done when a new copy of service code 1 1 7 for a service 116 becomes avail- 
able is use rmservice 605 to remove the old service 116 from server namespace 133 and then use either 
mkservice or putservice 627 to again place service 116 in server namespace 133. mkservice is used when 
client 107 and server 131 share a system name space and putservice is used when dient 107 and server 131 
belong to different system name spaces. Access by a dient 107 to a service node 1209 which is being changed 
can be avoided by the use of one many locking mechanisms. 

One way of taking full advantage of the possibilities opened up by servers 131 and services 116 is to in- 
dude a super server in the distributed system FIG. 7 shows a distributed system 701 with a super server 703. 
Like an ordinary server 131 . super server 703 has a file system 1 03(a) with service code 1 1 7(a..n) fbr a number 
of services. Super server 703 of course also has a server namespace 133 with nodes 1206 for the services. 
The difference between super server 703 and a server 131 is that super server 703 distributes services 116 
as well as executing them. 

One way in which such a super server 703 can be used is to distribute services 116 to dients 107. To re- 
ceive a service 116, the dient simply uses getservice 625 to get its copy of service code 117 for the service. 
Another way in which such a super server can be used is to distribute services 1 16 to other servers 131 . When 
super server 703 is being used this way. it responds to a request by dient 107 to execute a service 116 (j) 
whose code is induded in code 117(a..n) by downloading code 117(i) for the service 116 to a server 131(i) of 
servers 131 (0..m) (arrow 709) and setting server namespace 133 in super server 703 so that requests for 
service 116(j) are forwarded to server 131(i). The downloading is of course done using putservice 627. 

In another embodiment, super server 703 responds to a request for dient 1 07 for mkservice 605 for a ser- 
vice 116(i) by downloading service code 117(i) for service 116(i) to a server 131(k) in servers 131(0..m) and 
returning a handle 707 for service 116(i) in server 131(k) to dient 107. The handle is of course the name of 
server 1 31 (k) and the pathname of service 1 1 6(i) in server 1 3 1 (k)*s name space. Client 1 07 can then use handle 
707 to directly request server 131(k) to execute service 116(i). Super server 703's choice of a server 131(k) 
can of course be made from the point of view of load balancing. Similarly, if server 131(k) fails, dient 107 need 
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only repeat the mkservice request to get a new server 131. 

One area in which dients 107. servers 131. and services 116 can be used to particular advantage is the 
telephone switching system. The modern telephone switching system indudes a large distributed computing 
system. The components of the switching system often function as dients and servers for each other For ex- 
ample, when a call using an 800 number is made, the switching system must get the actual telephone number 
which currently corresponds to the 800 number from a data base. The switch which receives the 800 number 
IS a dient of the data base, which is a server for the 800 number. A number of properties of dients 1 07 servers 
131. and services 116 are particularly useful in the telephone switching system. There is first the fact that a 
service 116 may be added to or removed from a server 13Vs name space without interrupting operation of 
server 131: Then there is the fact that the system provides a number of techniques for load balandng among 
servers; further, the doseness of the relationship between a service 116 and the file system containing service 
code 117 may be varied: finally, services 116 may be easily propagated from server 131 to server 131 or be- 
tween a server 131 and a dient 107. 

One example of how dients 107. servers 131. and services 116 might be used in a telephone system is 
the following: d.ent 107 might be a programmable device such as an answering machine; server 131 might 
contain a service 116 which defined an interaction between a switdi and the answering machine: for instance 
the service 116 might permit the answering machine to request the number of the party whose call is being 
answered by the answering machine from the switch. The owner of the answering machine could call an 800 
number and specify that the service be provided to the answering machine at a given telephone number. The 
telephone system could respond to the call by causing server 131 to download the service 1 1 6 to the answering 
machine, which would then be able to perform the interaction defined by the service. 



5 Conclusion 



The foregoing Detailed Description has disdosed to those of ordinary skill in the art how to make and use 
a dient-server system whteh embodies the prindples of the inventions daimed herein. While the Detailed De- 
scription disdoses the best mode of implementing those principles presently known to the inventor, other im- 
plementations of the prindples are possible. For instance, in other implementations, communications between 
dient and server may employ mechanisms other than the remote procedure call. Similarly, in other implemen- 
tations, the server namespace may be flat instead of hierarchical. In other implementations, the service code 
may not indude the argument and result converters or the initialization code. Further, there may be applications 
where services are dynamically linked to dients or statically linked to servers. 



Claims 



1 . Apparatus for dividing execution of a program (117) between a caller process and a callee process running 
in a computer system, the apparatus being characterized by: 

means ( 1 005) for making a determination whether a process (1 07) executing the program Is a caller 
process or a callee process; and 
in the program, 
a caller portion (210) 
a callee portion (213). and 

a selecting portion (209) for selecting either the caller portion or the callee portion for execution 
by the process according to the determination, 

2. The apparatus set forth in daim 1 further characterized by: 

another process (131(a)): 

means responsive to executwn of the caller portion by the process (107) for sending a message 
(1019) requesting that the other process execute the program: 

means (1105) in the other process for receiving the message, making the determination in response 
thereto, and commencing execution of the selecting portion of the program. 

whereby the other process is able to execute the program as either a caller process or a callee process. 

3. The apparatus set forth in daim 2 further characterized in that 

the other process has access to namespace (133) means for relating a first name identifying the 
program to a location of a copy of the program; and 

the program further indudes an initialization portion (205) which is executed by the other process 
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in order to install the program in the namespace. 

4. The apparatus set forth in claim 2 further characterized by: 
a further process (131(b)). the further process having message receiving means like those in the 

other process; 

and wherein 

the other process has means (1023) responsive to the caller portion for sending a message (1019) 
to the further process requesting that the further process execute the program: and 

the means for receiving the message determines whether the callee portion is to be executed by 
the other process or the further process and causes the determination to indicate a callee process in the 
first case and a caller process in the second case. 

5. The apparatus set forth in claim 4 further characterized by: 

namespace means (133) accessible to the other process for relating a first name identifying the 
j5 program either to a location of a copy of the program or to a second name identifying the further process: 

and 

the means for receiving the message determines whether the called portion is to be executed by 
the other process or the further process in response to the namespace means. 

The apparatus set forth in daim 2 further characterized in that: 

the apparatus is implemented in a distributed computing system including a plurality of processing 
nodes (101); ^ 

the process and the other process execute on different ones of the nodes: and 
the means for sending a message sends the message from the node upon which the process is 
executing to the node upon which the other process is executing. 

7. The apparatus set forth in any of claims 2, 4. or 5 further characterized in that 

the means responsive to execution of the caller portion includes means (1 023) for receiving a return 
message from the other process with results of execution of the program: and 

the means for receiving the message indudes nrwans (101 6) for sending the return message with 
the results to the process. 
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8. The apparatus set forth in daim 7 further characterized in that: 

the means for sending a message, the means for receiving a message, the means for sending a 
result message, and the means for receiving a return message are provided by remote procedure call 
^5 means (1017). 

9. The apparatus set forth in daim 7 further characterized in that: 

the message and the return message employ a first representation of the data which is different 
from a second representation of the data employed in the computer system; 
^ the message indudes an argument value; 

the return message returns a result value; 
the program further indudes 
an argument converter portion (201) and 
a result converter portion (203); 
•^^ the means for sending a message and the means for receiving a message use the argument con- 

verter portion to convert argument values between the first representation and the second representation; 
and 

the means for sending a return message and the means for receiving the return message use the 
result converter portion to convert result values between the first representation and the second repre- 
50 sentation. 

1 0. A client-server system of the type wherein a server process (131) executing in a computer system provides 
a service for a client process (107), the dient-server system being characterized by: 

server name space means (1 33) employed by the server process to relate a name for the service 
55 to means (117) for providing the service, the name being part of a server namespace distinct from any 

system namespace provided to the server process by the computer system. 

service calling means (1016) employed by the dient process to send a message to the server which 
indudes the name for the service: and 
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service dispatching means (1105) employed by the server process for responding to the name for 
the service in the message by employing the server name space means to locate the means for providing 
the service and thereupon to employ the means for providing the service to provide the service to the 
dient. 

11. The dient-server system set forth in daim 1 0 further characterized by: 

one or more namespace manipulation services (1201) which the server process provides to the 
dient process and which the dient process employs to manipulate the server namespace. 

1^ The dient-server system set forth in daim 11 further characterized by: 

a make service service (605) of the namespace manipulation services which relates a new name 
in the server namespace to means for providing a new service. 
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@ Methods and apparatus for making and using distributed applications. 



\ A dient-server system for which applications 
programmers may easfly write services and in 
which a relationship between a server and a 
sen/ice may be changed without halting the 
server. Both dient and server have access to 
copies of code for the service. The code has two 
parts : a caller portion which requests a service 
and a callee portion which executes the service. 
State variables in the dient process and the 
server process detenmine which portion of the 
code is executed. This mechanism pemirts a 
server to forward execution of the service to 
another server. The code for the service is 
written using a template which relieves the 
applications programmer of the need to write 
specialized code. The server provides the dient 
with a server namespace which is distinct from 
the server's system namespace. The dient can 
locate a service by means of a service pathname 
in the system namespace. The server further 
provides the dient with namespace manipu- 
lation services which permit the dient to add 
services to and remove services from a server 
and otherwise to manipulate the server names- 
pace without halting the server. 
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